Data-driven Stylistic Humanlike Walk Synthesis

Human walk is a complex phenomenon and is hence difficult to model, even if the walking process is something we do every day without even thinking of it. A broad field of applications can nowadays be found for human motion synthesis: 3D animation and video games, medical application, sports, artistic performances, to name a few. Several approaches can be taken for the modeling of complex and high dimensional phenomena such as the motion modeling. In this thesis, we tackle the problem of model-based walk synthesis and highlight the strong parallelism that exists between speech and motion. We analyze how speech synthesis approaches, and more specifically Hidden Semi Markov Models (HSMM) taking the data dynamics into account, can be adapted to motion. Although the main scope of our work is to analyze how motion can be synthesized by applying probabilistic modeling techniques such as Hidden Markov Models, we also tested two methods which represent the motion space by a set of simpler functions, through Principal Component Analysis (PCA) or Fourier transform. In this work, two walk databases were recorded with an inertial motion capture system: the eNTERFACE’08 3D walk database (41 subjects walking at normal, slow and fast speeds and performing different direction changes) and the Mockey database (one actor performing 11 acted walk styles). These walk sequences were automatically segmented and used to train our models. The PCA approach was applied to the acted styles, and the inter-step variability of walk was modeled and taken into account during synthesis. Evaluations showed that the user were sensitive to any discrepancy in the synthesized motions, and that smooth transitions between the walk cycles were crucial for the motion naturalness. The Fourier time series decomposition also led to interesting results since two cosines per joint angle were sufficient to represent most styles. However, this method proved to be better suited for normal walk sequences than for exaggerated styles. We present an expressive gait synthesis system based on hidden Markov models (HMMs), following and modifying a procedure originally developed for speaking style adaptation, in speech synthesis. A large database of neutral motion capture walk sequences (eNTERFACE’08 database) was used to train an HMM of average walk. The model was then used for automatic adaptation to a particular style of walk using only a small amount of training data from the target style. The open source toolkit that we adapted for motion modeling enabled us to take into account the dynamics of the data and to model accurately the duration of each HMM state. We also address the assessment issue and propose a framework for qualitative user evaluation of the synthesized sequences. Our tests show that the style of the synthesized sequences can easily be recognized and look natural to the evaluators. These HMM models were used, through interpolation of their parameters, to synthesize styles or speeds which are combinations or exaggerations of the walk styles and speeds present in the training databases. Our synthesizer is also capable of generating walk sequences with smooth style transitions. A qualitative user